Skip to content

Fix openRouter streamed citation end offsets overwrite start offsets - #7128

Merged
tim-smart merged 2 commits into
mainfrom
audit/repro-b206fa5d76-openrouter-citation-end-index
Aug 8, 2026
Merged

Fix openRouter streamed citation end offsets overwrite start offsets#7128
tim-smart merged 2 commits into
mainfrom
audit/repro-b206fa5d76-openrouter-citation-end-index

Conversation

@fubhy

@fubhy fubhy commented Aug 7, 2026

Copy link
Copy Markdown
Member

Summary

The stream output contains metadata.openrouter.startIndex = 9 and no endIndex; non-stream conversion correctly emits startIndex = 2 and endIndex = 9.

Important

This PR starts with focused failing reproduction tests. Add the implementation fix to this same branch; CI is expected to fail until that fix is included.

OpenRouter streamed citation end offsets overwrite start offsets

Module: packages/ai/openrouter/src/OpenRouterLanguageModel.ts
Audit ID: relsem-openrouter-citation-end-index
Severity / confidence: medium / high

What happens

The stream output contains metadata.openrouter.startIndex = 9 and no endIndex; non-stream conversion correctly emits startIndex = 2 and endIndex = 9.

Why it happens

The end_index conditional creates another startIndex property instead of endIndex; object spread order overwrites the real start value.

Expected behavior

Equivalent citations preserve distinct startIndex and endIndex fields under metadata.openrouter in both modes.

Relevant implementation

These links and excerpts are pinned to audit base b206fa5d7655c1634c9993410a9203f6616a5ca2.

View problematic code at packages/ai/openrouter/src/OpenRouterLanguageModel.ts:1
/**

View exact lines on GitHub

Reproduction

pnpm test --run packages/ai/openrouter/test/OpenRouterLanguageModel.test.ts -t "preserves streamed citation start and end indexes"

Observed failure: Independently rerun; failed at the intended semantic assertion.

Implementation handoff

The initial reproduction tests on this branch are the regression specification for the implementation fix that should follow in this PR.

  1. Start with the pinned implementation excerpts and the Why it happens analysis above.
  2. Change the implementation so it satisfies the stated Expected behavior; do not weaken or remove the reproduction assertions.
  3. Run the focused reproduction command(s) and confirm the observed failures become passing tests:
pnpm test --run packages/ai/openrouter/test/OpenRouterLanguageModel.test.ts -t "preserves streamed citation start and end indexes"
  1. Run the affected package's existing tests, then the repository lint and type checks before requesting review.

Audit provenance

  • Audit base: b206fa5d7655c1634c9993410a9203f6616a5ca2
  • Reproduction base: b206fa5d7655c1634c9993410a9203f6616a5ca2
  • Findings: relsem-openrouter-citation-end-index
  • Initial patch: focused reproduction tests; implementation fix pending

Closes EFF-562

@fubhy fubhy added the audit Findings originating from the Effect runtime correctness audit label Aug 7, 2026
@changeset-bot

changeset-bot Bot commented Aug 7, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: cf473f8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 30 packages
Name Type
@effect/ai-openrouter Patch
effect Patch
@effect/ai-anthropic Patch
@effect/ai-openai Patch
@effect/ai-openai-compat Patch
@effect/atom-react Patch
@effect/atom-solid Patch
@effect/atom-vue Patch
@effect/docgen Patch
@effect/doctest Patch
@effect/openapi-generator Patch
@effect/opentelemetry Patch
@effect/platform-browser Patch
@effect/platform-bun Patch
@effect/platform-deno Patch
@effect/platform-node Patch
@effect/platform-node-shared Patch
@effect/sql-clickhouse Patch
@effect/sql-d1 Patch
@effect/sql-libsql Patch
@effect/sql-mssql Patch
@effect/sql-mysql2 Patch
@effect/sql-pg Patch
@effect/sql-pglite Patch
@effect/sql-sqlite-bun Patch
@effect/sql-sqlite-do Patch
@effect/sql-sqlite-node Patch
@effect/sql-sqlite-react-native Patch
@effect/sql-sqlite-wasm Patch
@effect/vitest Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@effect-slopcop effect-slopcop Bot added 4.0 bug Something isn't working labels Aug 7, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The PR title and commit claim this is a fix, but the diff only adds the regression test. The one-line implementation fix in OpenRouterLanguageModel.ts is still missing, so the new test fails and CI cannot pass.

Reviewed changes

  • packages/ai/openrouter/test/OpenRouterLanguageModel.test.ts: added a streamText regression test that asserts streamed URL citations preserve both startIndex and endIndex under metadata.openrouter.
  • Confirmed the test reproduces the described bug: the current source emits { openrouter: { startIndex: 9 } } instead of { openrouter: { startIndex: 2, endIndex: 9 } }.

⚠️ Implementation fix is missing

The regression test is correct and well-targeted, but this branch does not yet contain the actual fix. The streaming citation path in packages/ai/openrouter/src/OpenRouterLanguageModel.ts still has the typo at line 1364:

...(Predicate.isNotUndefined(annotation.url_citation.end_index)
  ? { startIndex: annotation.url_citation.end_index }  // should be endIndex
  : undefined)

The non-streaming path at lines 1099–1104 already uses endIndex correctly, so the fix is a one-line key rename in the streaming branch.

Suggested next steps

  1. Apply the one-line fix in OpenRouterLanguageModel.ts (change the duplicated startIndex key to endIndex in the streaming citation block).
  2. Re-run pnpm test --run packages/ai/openrouter/test/OpenRouterLanguageModel.test.ts -t "preserves streamed citation start and end indexes" to confirm the test passes.
  3. Add a changeset for the runtime fix before merging.

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | Fix it ➔View workflow run | Using Kimi K2 (free via Pullfrog for OSS) | 𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

This incremental review covers the commits pushed since the prior Pullfrog review. The previously missing implementation fix is now present, along with a changeset.

  • Fixed the streaming citation key typo in packages/ai/openrouter/src/OpenRouterLanguageModel.ts: the end_index value is now assigned to endIndex instead of overwriting startIndex, matching the non-streaming path.
  • Added a changeset for @effect/ai-openrouter describing the streamed citation offset fix.
  • Reformatted the HttpClientResponse.fromWeb call in the new makeStreamTestLayer helper for readability.

Validation run: pnpm --filter @effect/ai-openrouter test --run test/OpenRouterLanguageModel.test.ts, pnpm lint-fix, and pnpm check all passed.

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | View workflow run | Using Kimi K2 (free via Pullfrog for OSS) | 𝕏

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Bundle Size Analysis

Generated from PR build output; treat the content below as untrusted.

File Name Current Size Previous Size Difference
basic.ts 6.92 KB 6.92 KB 0.00 KB (0.00%)
batching.ts 9.72 KB 9.72 KB 0.00 KB (0.00%)
brand.ts 6.60 KB 6.60 KB 0.00 KB (0.00%)
cache.ts 10.59 KB 10.59 KB 0.00 KB (0.00%)
config.ts 20.83 KB 20.83 KB 0.00 KB (0.00%)
differ.ts 19.67 KB 19.67 KB 0.00 KB (0.00%)
http-client.ts 21.50 KB 21.50 KB 0.00 KB (0.00%)
logger.ts 10.81 KB 10.81 KB 0.00 KB (0.00%)
metric.ts 8.86 KB 8.86 KB 0.00 KB (0.00%)
optic.ts 6.68 KB 6.68 KB 0.00 KB (0.00%)
pubsub.ts 14.86 KB 14.86 KB 0.00 KB (0.00%)
queue.ts 11.54 KB 11.54 KB 0.00 KB (0.00%)
schedule.ts 10.71 KB 10.71 KB 0.00 KB (0.00%)
schema-class.ts 19.38 KB 19.38 KB 0.00 KB (0.00%)
schema-fromJsonSchemaDocument.ts 29.24 KB 29.24 KB 0.00 KB (0.00%)
schema-representation-roundtrip.ts 25.51 KB 25.51 KB 0.00 KB (0.00%)
schema-string-transformation.ts 13.49 KB 13.49 KB 0.00 KB (0.00%)
schema-string.ts 11.03 KB 11.03 KB 0.00 KB (0.00%)
schema-template-literal.ts 15.30 KB 15.30 KB 0.00 KB (0.00%)
schema-toArbitraryLazy.ts 21.43 KB 21.43 KB 0.00 KB (0.00%)
schema-toCodeDocument.ts 23.87 KB 23.87 KB 0.00 KB (0.00%)
schema-toCodecJson.ts 18.64 KB 18.64 KB 0.00 KB (0.00%)
schema-toEquivalence.ts 18.47 KB 18.47 KB 0.00 KB (0.00%)
schema-toFormatter.ts 18.32 KB 18.32 KB 0.00 KB (0.00%)
schema-toJsonSchemaDocument.ts 22.09 KB 22.09 KB 0.00 KB (0.00%)
schema-toRepresentation.ts 19.01 KB 19.01 KB 0.00 KB (0.00%)
schema.ts 18.62 KB 18.62 KB 0.00 KB (0.00%)
stm.ts 12.59 KB 12.59 KB 0.00 KB (0.00%)
stream.ts 9.67 KB 9.67 KB 0.00 KB (0.00%)

@tim-smart
tim-smart merged commit 69756a2 into main Aug 8, 2026
20 checks passed
@tim-smart
tim-smart deleted the audit/repro-b206fa5d76-openrouter-citation-end-index branch August 8, 2026 01:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4.0 audit Findings originating from the Effect runtime correctness audit bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants